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Introduction 


Assessing a student’s ability to be successful in courses and programs is an important and 
necessary aspect of student success. Two major practices exist to predict a student’s likelihood of succeeding 
in a course or program: 1) Successful completion of prerequisite or advisory courses (as documented on 
transcripts) and 2) the assessment for placement process. These two methods are presumed to be mechanisms 
that ensure that a student has acquired the knowledge and skills necessary for success. 

Presuming a student is prepared for a course through the completion of a prerequisite course is a rather 
straightforward process; however, placing a student using an assessment for placement process is necessarily 
more complicated, as such placements cannot be made based on assessment test scores alone. Some students 
may possess necessary course or program skills but have difficulty demonstrating those skills on standardized 
tests or fail to prepare adequately for an assessment test. For this reason. Title 5 §55502(i) clearly mandates 
that California community colleges use multiple measures in their assessment processes: ‘“Multiple measures’ 
are a required component of a district’s assessment system and refer to the use of more than one assessment 
measure in order to assess the student” [emphasis added]. The requirement to use multiple measures is 
reiterated in Title 5 §55522(a): “When using an English, mathematics, or ESL assessment test for placement, 
it must be used with one or more other measures to comprise multiple measures.” 


While multiple measures have always been required by Title 5, adequate research into the accuracy of these 
measures has not been readily available to inform educational decisions. Individual colleges have made 
various decisions regarding the use of subjective measures and have therefore reported differing experiences. 
While colleges are required to employ assessment tools that have been validated, no mandate exists for a 
corresponding effort to validate the application of multiple measures. This paper addresses the broader issue 
beyond simply evaluating a transcript for previous coursework or limiting placement based on an exam; 
it examines the use of multiple measures in addition to placement tests as a way to improve the overall 
assessment of students’ abilities. 
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Title 5 §53200 gives academic senates the responsibility for making recommendations about academic and 
professional matters concerning “standards or policies regarding student preparation and success.” The 
intent behind prerequisites and placement processes, including the selection and application of multiple 
measures, is to ensure or enhance student success through proper preparation. Therefore, academic senates 
must be directly involved and play a leading role in facilitating and developing recommendations about 
assessment processes and the use of multiple measures at both local and state levels. 

The concept of applying multiple measures for placement is often misunderstood by local colleges, and 
data are sometimes difficult to collect. Even within a single college placement practices may vary among 
different disciplines. Multiple methods and placement practices were summarized by Regional Education 
Laboratory (REE) in 20 IL A survey was implemented to examine current practices and applications of 
multiple measures. Of the 1 12 colleges in the California Community College (CCC) System, 59, or just over 
half provided survey information about multiple measures (See Appendix A). The survey noted that only 
48 of the 59 responding colleges reported how they used multiple measures, and 34 of the colleges reported 
using a “weighted score” of placement tests and then adding or subtracting points for multiple measures. 
Twelve colleges reported relying most heavily on qualitative data to direct placement decisions, placing less 
consideration on placement test scores. REE reported that weighting of multiple measures varied widely and 
that only a few colleges used regression analysis to predict success. 

In this paper, “use of multiple measures for placement,” or simply “multiple measures,” refers to a process in 
which colleges rely on more than a single factor to determine student readiness for a course or program. The 
purpose of this paper is to do the following: 

• review the value of and reasons for using multiple measures in California community colleges for 
placing students into the curriculum; 

. address the role of the academic senate, discipline experts, and counseling faculty in multiple 
measures placement; 

• provide guidance regarding best practices for implementing multiple measures in order to improve 
placement accuracy. 

In addition, the paper will explore the implications of multiple measures on current issues involving efforts 
to implement a common assessment across the state, including unresolved issues of portability of assessment 
for placement results, accuracy and reliability of assessment, and local autonomy regarding assessment and 
placement decisions. 
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Regulatory Framework 

Regulatory Guidelines for Multiple Measures 

Title 5 §§55502(1) and 55522(A) explicitly require the use of multiple measures in assessment 
for placement. Thus, the question for California community colleges and districts is not whether to use 
multiple measures, but which measures to use and how to apply them. 

Colleges have latitude in what types of multiple measures may be used in their placement process. In addition 
to assessment tests. Title 5 §55502(i) notes that additional measures may include “interviews, holistic 
scoring processes, attitude surveys, vocational or career aptitude and interest inventories, high school or 
college transcripts, specialized certificates or licenses, education and employment histories, and military 
training and experience.” Furthermore, Title 5 §55522(a)(2) allows the Chancellor’s Office to “identify other 
measures of a student’s college readiness that community college districts may use for student placement 
into the college’s curriculum.” This section of Title 5 has permitted some colleges to explore the use of high 
school transcripts for placement, as well as the scores earned through the California State University’s (CSU) 
Early Assessment Program. 

As noted in these Title 5 sections, the initial assessment process must consist of multiple measures, meaning 
that all the various measures must be collected and evaluated prior to determining the student’s placement. 
Colleges that rely only on assessment tests for initial placement but then allow other measures to be 
considered on appeal of the decision are not employing a multiple measures approach to placement. Instead 
they have a single measure placement approach with a multiple measures appeals process, which is not in 
alignment with the law. 

Although assessment processes are most commonly employed to place students in appropriate English, 
mathematics, or English as a Second Language (ESL) courses, a college may have assessment processes 
for other sequential courses in the curriculum (e.g.. Chemistry). Since Title 5 §55502(i) indicates that 
multiple measures are a required component of a college’s or district’s assessment system, colleges must also 
implement multiple measures of assessment for any subject in which they have a placement process, and 
these measures should be determined using data that provide knowledge about each measure’s usefulness 
and accuracy. 

Regulatory Discussion on Assessment Tests 

Of the set of possible multiple measures. Title 5 places the most stringent guidelines on assessment tests 
for placement. Using guidelines prepared by the Chancellor’s Office, disfricfs and colleges musf validate all 
assessment for placement tests to ensure that the tests are being used in a proper manner and that the tests 
show little or no cultural or linguistic bias (Title 5 §55522(a)(l)). In general, with minor exceptions, the 
following mandates and restrictions apply to all such tests: 
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• Assessment test procedures must be clearly communicated to students, including the availability of 
sample tests, how assessment test results will inform placement decisions, and any limits that the 
college or district places on retakes of the assessment test. [§55522(b)] 

• Assessment tests must be approved by the Chancellors Office. [§55522(c)(l)] 

• Assessment tests may not be used in a manner or for a purpose other than that for which it was 
developed or otherwise validated. [§55522(c)(2)] 

• Assessment tests may not be used to deny admission to a college. [§55522(c)(3)] 

• Assessment tests may not be used to exclude students from any particular course or educational 
program, except that districts may establish appropriate prerequisites. [555522(c)(4)] 

Although Title 5 contains no other specific guidelines about reviewing and evaluating other assessment 
measures. Title 5§55522(c)(5) includes a blanket prohibition that no “Student Success and Support Program 
practice which has the effect of subjecting any person to unlawful discrimination” is permitted. Any 
assessment test that contains culturally or otherwise biased content or language is therefore in violation 
of state regulation. The Chancellors Office Assessment Workgroup, which reviews and approves all locally 
developed or proposed assessment instruments, focuses on ensuring that the instruments do not reflect any 
discrimination or bias. However, colleges and districts should scrutinize their assessment for placement 
processes locally to guarantee the absence of prejudicial language and content, not only to comply with Title 
5 but simply as good practice for the sake of students. Furthermore, the application of multiple measures 
should be reviewed for any potential bias. Bias is not intuitive and is only evident when placement results are 
examined using data that are disaggregated, and even that examination may not be sufficient to identify bias. 
If, for example, a high school experience that is only available to certain populations of students weighted 
heavily in the placement process, this practice should be reviewed for potential adverse impacts. 


The Case for Multiple Measures 

Limiting assessment to a placement test, an exam with content and skills questions from several courses 
in a curricular sequence, represents a single and potentially unsuitable measure of student preparation for 
college coursework. Likewise, assessment of foundational skills such as writing, reading, and computation, 
may be problematic if limited to measures such as transcript evaluation, employment history, interviews, 
and attitude surveys because, among other factors, students may have returned to college after a long break 
or because such measures involve a level of self-reporting that, unintentionally or not, may not accurately 
reflect students’ preparation. 

The drawbacks of relying exclusively on a single, high-stakes exam or test for placement decisions are well 
known. Many have noted that since the placement tests themselves are fairly brief, they are not able to 
delve deeply enough into the assessed subjects to provide a more complete assessment picture. Another 
problem with placement tests is that even the best students may do poorly taking an exam on content they 
may not have engaged with for months or even years. Poor scores for these students lead to the problem of 
underplacement, the placement of students in courses considerably below their true knowledge and skill 
level. On the other end of the spectrum, since many placement exams rely on multiple choice items, students 
who are strategic about guessing and eliminating wrong answers may achieve high scores without having 
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a solid grasp of a subject’s underlying principles. High scores for such students may lead to overplacement, 
the placement of students in courses beyond their skill set and, consequently, in which students are unlikely 
to succeed. 

Research suggests that one reason to use multiple measures is to increase the predictive ability of the 
placement process. In other words, applying multiple measures can increase the accuracy of the placement 
process over a single placement test. Findings from a recent predictive model study regarding the use of 
multiple measures that examined placements and success rates for 42,000 first-time entrants to a large urban 
community college system suggest that the use of multiple measures when determining student placements 
lower the number of students that need remediation (up to 12%) by placing them in the correct courses and 
at the same time increase the success of these students in the courses that they were placed (Scott-Clayton, 
2012). Scott-Clayton (2012) also notes that the benefits of using multiple measures rather than relying on 
assessment tests alone are particularly strong for English placements because the use of assessment tests 
alone seems to be more accurate for math placements than for English. The study also reported that an 
estimated 25-33% of students are incorrectly over or under assessed when additional measures are not 
included in the placement process. Other studies have reported similar problems with assessments. 

Clayton, Crosta, and Belfield (2012) note that “roughly one in four test-takers in math and one in three 
test-takers in English are severely mis-assigned, with severe under-placements in remediation much more 
common than severe over- placements in college-level coursework” (p. 4). 

If misplacement claims made by Scott-Clayton, Crosta, and Belfield (2012) and others are accurate, the 
consequences for community colleges that fail to include high quality, validated multiple measures in the 
placement process are significant. In places like California, which has 112 community colleges across the 
state, an incredibly large number of placement tests are given in a single year, so research suggests that the 
number of misplacements is likely to be quite high. The following placement test data from Eall 2010 offers 
a sense of the potential magnitude of the problem: 


Subject 

Number of Placement Tests Given 

Math 

350,129 


English-Writing 319,892 


English-Reading 282,936 


ESL-writing 

20,925 


ESL-reading 

21,560 


ESL-intergrated 

31,297 



While this chart does not indicate how much misplacement occurred in each category, research suggests 
the number of misplacements may have been sizable. The large number of total placements in California 
suggests that error rates of 25-33% impact a considerably large number of students. Eor this reason, we 
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must be more critical of the accuracy of our colleges’ assessment data and more curious about how our 
community colleges use high quality, validated multiple measures in their assessment processes. 

Inaccurate assessments resulting from the use of placement tests without multiple measures may contribute 
to poor persistence and success rates of basic skills students. We know that students enrolled in basic skills 
courses often do not move through English and mathematic basic skills sequences and complete transfer 
level courses in high numbers. In English, for example, of those who assessed at below transfer level, “the 
largest proportion assessed at two levels below transfer level, and about 38% of this proportion succeeded 
in completing transfer level English,” and in mathematics, “of those who assessed at below transfer level in 
mathematics, the largest proportion assessed at three levels below transfer level, and 14% of this proportion 
succeeded in completing transfer-level mathematics” (Skinner, 2012). If 25-33% of students who assess are 
misplaced, then the misplacements may be impacting students’ ability to pass courses, learn skills, and 
persist in high numbers. 

Since the use of placement tests in isolation may result in students being inaccurately placed into remediation 
when they were prepared for coUege-level work, more accurate placements may also result in various direct 
benefits for these more prepared students. Scott-Clayton, Crosta, & Belfield (2012) noted that 

prepared students who are assigned to remediation may garner little or no educational benefit, but 
incur additional tuition and time costs and may be discouraged from or delayed in their degree plans. 
Indeed, several studies using regression-discontinuity (RD) analysis to compare students just above 
and just below remedial test score cutoffs have generally found null to negative impacts of remediation 
for these ‘marginal’ students, (p. 2) 

While the utility of remediation is a complex subject that may produce various perspectives and conclusions, 
certainly no student who truly has no need of remediation should be required to complete unnecessary 
instruction, and the use of multiple measures may help to reduce the number of students placed into this 
situation. 

Another reason that correct placement is essential is that remediation is expensive, both in terms of fiscal 
costs and the time it takes students to complete their degrees. The fiscal cost of remediation has been 
estimated to be nearly $7 bhlion dollars per year in the United States (Scott-Clayton, Crosta, & Belfield, 
2012). Many other costs related to the impact of remediation on students are not included in this estimate. 

A final important reason to use multiple measures in the placement process is that colleges may use the 
improved assessment data to create better planning processes. By improving the accuracy of placement 
process, colleges can collect better data and use that data to impact other important processes from 
scheduling to educational planning. Colleges often struggle to determine how many sections of a particular 
course should be offered at each level of a curricular sequence, for example. More accurate assessment data 
may be one of the keys to improving these planning processes, and the inclusion of multiple measures in our 
assessment processes is an important step toward the goal of improving the accuracy of assessments. 
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Pros and Cons of Individual Assessment Measures 

The following table looks at a collection of pros and cons of some commonly used multiple measures. A 
complete chart is found in Appendix B. 




Measure 

Pros 

Cons 

Additional notes 

Assessment using a standardized test 

Placement tests 

Standardized California Standards 
test 

Early Assessment Program Test 
(EAP) 

Standardized and 
comparable among 
students that have taken 
the test. 

A single high stakes 
test may not reflect 
all the student's skills 
and abilities and may 
introduce testing bias. 

Requires cut score validation. 

Accuracy may vary with test used. 

Alignment with curriculum may 
affect content validity. 

Prior academic achievement 

GPA (self-reported) 

Transcripts 

AP - Advance Placement course 
completion or test scores 

Highest level course success 

Provides an aggregate 
measure of student 
achievement and 
motivation. 

Concerns exists regarding 
accuracy or validity of 
grades and course work, 
especially when self- 
reported. 

May be affected by the recency 
with which the skills or knowledge 
were acquired. 

Not comparable across a large 
population of students from 
different schools. 

Demographics and socioeconomic status 

Age 

Full-time/ part-time status 

Number of hours employed 

First generation status (highest 
level of parental education) 

Socioeconomic status (federal Pell 
Grant or Board of Governor's Fee 
Waiver recipient) 

Evidence suggests that 
full-time status results in 
better success than part- 
time status. 

Specific evidence in this 
area may not take into 
consideration other 
factors regarding full- 
time vs. part-time such as 
the economic or family 
conditions that allow full- 
time enrollment. 

Data are inconsistent here, at times 
showing better success in younger 
age groups and other times in 
older students. 

Direct correlations are currently not 
objectively measurable. 

Student profiling may occur. 

Inequitable treatment of students 
inevitable. 


Affective measures 

Motivation (self-reported) 
Declaration of a major 

Assessment of motivation 
level. 

Self-reported information 
is very subjective. 

Accuracy issues arise in the 
selection, recording, and changing 
of a major. 

Other measureable factors that contribute to academic success 

Student education plans 
developed in 8th grade or high 
school that continue through 
college (called "programs 
of study" in some districts) 
representing evidence of 
motivation 

Data has correlated these 
activities with success. 

These variables are broad 
and depend upon rigor, 
content, and alignment 
with student goals. 

The value of this data may vary 
with the college. 

Employment history 
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Relationship between Multiple Measures, Prerequisites, and Assessment for 
Placement 

One way in which multiple measures may he used is to verify attainment of skills and knowledge expected 
in a prerequisite. As a condition of enrollment, a prerequisite is intended to specifically identify the skills or 
body of knowledge a student must have in order to be successful in the target course, and “the determination 
of whether a student meets a prerequisite shall be based on successful completion of an appropriate course or 
on an assessment using multiple measures” (§55003). Further, the ASCCC asserts that “students are placed 
into courses based on their success in prerequisite courses or an assessment process that involves the use of 
standardized tests in conjunction with other measures likely to effect the student’s performance (ASCCC, 
2011). In other words, all placement decisions require the use of multiple measures. 

Although prerequisites have long been a useful tool with which faculty can increase the likelihood of student 
success in their courses, recent events have brought a renewed sense of importance and interest in their use. 
Notably, the final recommendations from the Student Success Task Force (2011) include Recommendation 
3.3, incentivizing students to “begin addressing basic skills deficiencies in the first year.” Data from Bakersfield 
College (2013) support the fact that students who take their remedial courses in the first term are more 
successful in all subsequent courses. 

Multiple Measures should contribute to the success of placement, especially in prerequisite courses. However, 
placing prerequisites on courses does not guarantee the successful of completion of target course unless the 
curriculum is aligned. 

Making Placement Decisions with Multiple Measures 

All measures used for placement should be scrutinized and adopted based on available data rather than 
ease and portability alone. One way to address this issue is to have each college create a process to validate 
all measures, not just standardized assessments. However, some colleges may not be readily able to provide 
logistical regression or statistical models for their own student populations. For this reason, to some extent 
a statewide examination of these individual measures and their reliability should be conducted and made 
accessible to individual colleges. However, recognition of the unique curriculum alignment factors and local 
populations places a heavy responsibility on the local colleges to review and use the data appropriately. 

Locally, colleges, in consultation with their academic senates, should have written processes for validation 
and placement that involve discipline-specific faculty and student affairs areas. Discussions should, at a 
minimum, include content of courses, level of rigor, and college-level skill requirements such as writing, 
research, expectations, and hours of work outside of class. The process should include built-in mechanisms 
to collect data relevant to placement and success. This work usually requires the involvement of a researcher 
who is part of the discussions from the early stages of development. 

Multiple Measures in Conjunction with a Common Assessment Test 

In recent years, many individuals and interested groups have called for placement results that are portable 
among the colleges. The Board of Governors of the California Community Colleges has asserted the value 
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of portable placement results, and Recommendation 2.1 of the Student Success Task Force (2011) stated that 
“Community colleges will develop and implement a common centralized assessment for English reading and 
writing, mathematics, and ESL.” (p. 25) A common assessment and individualized cut scores can to some 
extent facilitate the goal of establishing portable placement results, but the integration of such a common 
instrument with multiple measures is a challenge. One goal of this paper and the ongoing research is to 
explore a multiple measures approach that is both portable and accurate. 

A system of portable placement decisions could benefit the students and the state in various ways. Currently, 
each California community college selects and implements its own placement system. While certain popular 
instruments may be common to multiple districts, great variety still exists. Moreover, even if districts share 
a particular assessment instrument, they may not accept each other’s placement decisions: each district 
may interpret and apply test results differently and may use different multiple measures to make placement 
decisions. Students, therefore, are often forced to re-test and are assigned a new placement result when they 
move among colleges. A system of portable placement would benefit students and create greater consistency 
by eliminating the need for retesting and re-placement at each college. Such a system would be especially 
useful to the many students who take courses at more than one college and might enable easier student 
movement among community colleges. In addition, community colleges and the state would experience 
cost savings if the number of assessment tests taken could be reduced by making placement portable and 
minimizing or eliminating the need for retesting. 

California already has some tools in place which may help to facilitate the creation of a system of portable 
placement results. Work on developing a common assessment instrument is already underway. Curricular 
comparability through use of the CB 21 rubrics and the Course Identification Numbering System (C-ID) 
system may also allow for or encourage a more universal placement system. However, the application of 
multiple measures must also be a part of any discussion that hopes to achieve portable placement results. 

Agreement on the use of multiple measures in portable placement results for the California community 
college system could take various forms. One possibility is that colleges would reach concurrence on a 
consistent system of multiple measures that all institutions would accept. Such discussions might best begin 
regionally, as colleges in a given area might compare the measures they are using and find that they are 
applying different processes that achieve the same or similar results. In the absence of statewide consensus 
and portability, the creation of regional agreements could provide significant benefit to students, as many 
students who move among various colleges would be most likely to do so within a given geographic area, 
and institutions would still experience specific cost savings through a reduction of re-testing. Such regional 
agreements might also, over time, lead to broader discussions that create state-level portability. 

Alternatively, colleges might develop their own applications of the common placement instrument and of 
multiple measures and then agree to accept and trust the results of other institutions. In such an instance, the 
use of multiple measures may help community colleges to tailor the common assessment instrument to meet 
local needs. Local community colleges could control the use of validated local multiple measures that they 
determine are relevant for student placement in their communities. If colleges agree to honor the placement 
results that student bring from other institutions, then a system of portable placement would result without 
requiring specific agreement on the application of the assessment instrument or of multiple measures. 
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However, the concept of portable placement does raise issues for many within the California community 
college system. Certainly colleges within the system strive toward comparability in curriculum and standards 
through such projects as the CB 21 rubrics, the C-ID system, and the Transfer Model Curricula for the 
Associate Degrees for Transfer. Nevertheless, with 112 institutions and nearly 2 Vi million students spread 
out across more than 150,000 square miles, each district and even colleges within a district have their own 
unique populations to serve and issues to address. Portable placement is challenging when one considers the 
diversity of course offerings for unique student populations and innovations for student course completion. 
Based on such factors as compressed calendars, accelerated coursework, and different instructional 
modalities, individual multiple measures may take on different meaning even within a given discipline. 

Given the wide variety of communities and cultures found throughout the state, the concept of local control 
over decision-making is a fundamental value to many within the community college system. Any discussion 
of common assessment or portable placement must be weighed against the importance of local control and 
should strive to respect this concept to the greatest extent reasonable while still serving the needs of both 
students and the state as a whole. 

Collaboration in the local determination of Multiple Measures and 

PLACEMENT PROCESSES. 

Whereas the regulatory framework of Title 5 provides colleges and districts with information regarding 
what is required and prohibited with respect to multiple measures assessment, it does not give guidance to 
colleges and districts on what measures to use or how to interpret individual or aggregated measurements. 
The professional judgment of discipline experts and counseling faculty is necessary to fashion a multiple 
measures assessment process that maximizes students’ likelihood of success in the courses in which they are 
placed. 

While research on community college placement processes supports the use of multiple measures, integrating 
multiple measures into placement processes is not without costs and challenges. Multiple measures need to 
be carefully examined and matched to the needs of particular college communities so that local colleges 
can extend resources and time to validate their measures. Discipline experts and counseling faculty, as well 
as the local academic senate, need to ensure that the multiple measures selected by their college contribute 
meaningfully to the placement process. All parties involved with placement of students need to evaluate 
the ways in which the use of multiple measures fits into existing placement processes and to incorporate a 
periodic review on the efficacy of the multiple measures. 

Academic senates, which have responsibility for making recommendations about academic and professional 
matters regarding “standards or policies regarding student preparation and success,” play an important role 
facilitating and developing recommendations about multiple measures assessment. The purpose of this 
section is to provide context and structure for academic senate leaders, discipline experts and counseling 
faculty as they develop local multiple measures policies and procedures. 

As discipline experts, the faculty in the areas being assessed have several roles to play in the assessment for 
placement process. In developing the college’s placement system, discipline experts can determine the skill 
sets that are needed for success in target courses and with research partners can determine the best use of 
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multiple measures along with cut-off scores for assessment tests. Part of the role of discipline experts is to 
discuss the result of the validation data and cut-off scores and fo ensure fhaf placement decisions result 
in student success by participating in the evaluation of placement processes. If the college is basing initial 
placement decisions on information other than an assessment test, such as using students’ high school grades, 
discipline experts should take responsibility for determining how that information is applied for placement. 
Faculty will need to consider how to best use available, validated data. Content validity is an important aspect 
of any test such that discipline experts need to review and interpret data. Student behaviors and sophistication 
are important factors in success, and therefore student affairs professionals should help interpret data. The 
data alone, produced by a researcher, do not provide the interpretation and application. For this reason, 
collaboration and review are essential. Discipline experts must be actively involved in discussions to help 
validate the alignment of the methods and the results of the assessment process to the requirements of the 
coursework and must not allow the local researcher to make decisions or reach conclusions alone (Lagunoff, 
Michaels, Morris, & Yeagley, 2012). In addition, the reliability of all measures should be routinely evaluated 
based on factual outcomes of the placement decisions, including data indicating what students were placed 
in what courses by what measures and with what rates of success. Consistent and ongoing evaluation of the 
process and its results is crucial to ensuring the validity and accuracy of the placement system. 

In some cases, the primary roles of discipline experts may involve establishing and evaluating the assessment 
system, while initial individual placement decisions are established by the college’s assessment staff based 
on tesf resulfs. In ofher cases, such as placing students based on English faculty reading placement writing 
samples, discipline experts may themselves determine the initial placement of a student. Regardless of the 
process and whether their role includes establishment and evaluation of the system or direct involvement in 
placement decisions, discipline experts should take direct responsibility for the initial placements assigned 
to students. 

Approval of a multiple measures process can include: the weighting for each of the measures based on 
local research; decision matrices clearly demonstrating the roles of decision makers; discipline experts 
determining prerequisites; discussions with research, admissions, counseling and discipline experts setting 
cut scores for assessment tests; and other measures based upon discussions with counseling, discipline 
experts and research. 

Counselors must be involved in the placement process before final placements are determined, not j ust during 
the appeals process. As identified in the Standards of Practice for California Community Colleges Counseling 
Faculty and Programs (ASCCC, 2008), “academic counseling services include assessment using multiple 
measures and diagnosis of students’ academic abilities, disabilities, strengths and weaknesses.” (p. 4) As a 
result, counseling faculty are trained to apply those measures and then make placement recommendations 
based on a combination of assessment results and the institution’s use of multiple measures. Counselors 
need to know how to work with students in exploring information about their past educational experience, 
college readiness, possible disabilities, and academic skills. These questions become crucial when a student 
places below transfer-level English and math. 


Once initial placement based on discipline experts expertise and multiple measures adjustments by trained 
counseling faculty have been determined, collaboration between discipline experts and counseling faculty 
is crucial. A college must work to effectively ensure that the application of multiple measures by counseling 
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faculty is resulting in placement decisions that are comparable to those that would be made by discipline 
experts and that the use of multiple measures does not result in placements inconsistent with the decisions 
and standards of discipline experts. While clear guidance can be developed for how to factor in external 
exam scores into placement decisions, counseling and discipline experts must collaborate to establish a 
common understanding as to how other measures factor into such determinations and what steps are taken 
to verify that the placement process is effective. No single system of collaboration will work for all colleges, 
but each institution must work to create a system in which multiple measures are meaningfully applied, 
counseling expertise is respected, and the judgment of the discipline experts is utilized. 

In compiling the necessary data for both establishing and evaluating an accurate assessment for placement 
system, college researchers also have a significant role to play. The knowledge and training of researchers 
can help to ensure that information used in the assessment system is valid, complete, and clear. However, 
the data alone, produced by a researcher, do not provide the interpretation and application that faculty can 
provide. Student services professionals and discipline experts must interpret the data, often with guidance 
from the college researcher, and take responsibility for final decisions regarding the system. The college 
researcher is an important member of the collaborative effort necessary to create and evaluate the placement 
system, but the final decisions regarding the system should remain in the hands of faculty experts. 

While the use of multiple measures is supported by the research on community college placement processes, 
integrating multiple measures into placement processes is not without costs and challenges. Multiple 
measures need to be carefully examined and matched to the needs of particular college communities, so 
local colleges can extend resources and time to validate their measures. Discipline experts, counseling 
faculty, and the local academic senate need to be certain that the multiple measures selected by their college 
contribute meaningfully to the placement process. All parties involved with placement of students need to 
evaluate the ways in which the use of multiple measures fits into existing placement processes and to build in 
periodic review of how well the placement process results in student success. There are varying approaches 
to collaboration regarding assessment and multiple measures and two possible scenarios are presented in 
Appendix C. 
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Recommendations eor Local Senates 

See checklist of recommendations in Appendix D 

. Ensure that assessment procedures and the way placement decisions are made are clearly 
communicated to students. Students should he informed about the entire set of multiple measures 
that are being used to assess their level of knowledge and skill and how those multiple measures 
will be analyzed. 

. Ensure that multiple measures are applied consistently for all students. 

• Collect multiple measures before students complete assessment tests or as part of the assessment 
test process so that multiple measures are being applied to all students who are assessed, not just 
those who appeal their assessments. 

• Use measures that have a high degree of predictive validity. This may require longitudinal analysis of 
the predictive value of specific measures within service areas. Eor example, some communities may 
find relatively high predictive validity for high school math grades whereas in other communities 
that measure may be less useful. 

. Involve discussions by the local senate and discipline experts at each college. 

• Create a local selection of validated measures policy and data. 

. Include periodic review of multiple measures assessment policies. 

. Provide discipline experts and counseling faculty with information on why certain multiple 
measures have been selected for use at the college and the role that multiple measures can play in 
accurate placement. 

• Strive to produce an objective process and carefully examine the use of local measures that may be 
overly subjective, such as interviews. 

. Make weighting of multiple measures transparent and research based. 

• Consider a regional consortium among the counseling faculty and discipline experts to discuss 
how assessment outcomes might be portable and accurate. 
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Conclusion 

The role of multiple measures is to enable an institution to look at a variety of student characteristics to 
successfully place each student in the courses best suited to his or her skills and abilities. Multiple measures are 
clearly mandated, and yet they are a challenge to implement. Institutions must rely upon multiple measures 
rather than a single test or any other single measure because research has shown that no single measure is 
fair or adequate to make the complex prediction about a student’s success. Data collected on the practices 
of CCCs demonstrate that no real consistency exists in what measures are used or how multiple measures 
are implemented at different colleges. Other than the common use of commercial tests, the similarities of 
multiple measures among the CCCs is very small. 

While the necessity to use more than one measure is undeniably supported by Title 5 requirements and by 
research, the determination of which set of measures to use is much more complicated. The value of the 
multiple measures vary when they are applied to different courses, diverse colleges, and dissimilar students. 
Once the multiple measures are decided upon, the application of the measures varies depending upon the 
policies of the colleges. Some colleges rely wholly upon discipline experts, while other colleges rely more 
heavily on counselors. Some colleges value objective measures, other colleges see the importance of including 
subjective measures with objective data, and still other colleges allow student self-placement. 

The low success rate in the classes where placement is most controlled should challenge us to sharpen our 
use of multiple measures and to do more research, to work more collegially, and to make better decisions 
based upon more data. Because this work is not simple, the task demands collaborative thinking and 
cooperation between student services and instruction. The significant costs to the student and the institution 
should lead colleges to pursue better research targeting the viability of multiple measures and to make a 
commitment to develop policies that evaluate the success of multiple measure placements. Many of these 
issues are being studied as this paper is being finalized. Prompt follow-up to this paper may be necessary, 
with more published information describing statewide studies examining the use of high school data and 
other multiple measures. 
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Appendix A 

Current Commonly Used Multiple Measures Summarized from the CCCCO Survey as Analyzed by WestEd 
2011 (N= 59 of 112 colleges) 


Tests used for placement and the disciplines they are used 
for 

Percent of respondents using this 
Total 57 of 59 (85%) colleges used tests 

Accuplacer - English, Reading, Writing, ESL 

68% (but varies depending on discipline it is used for e.g. 
only a few use this for ESL) 

CELSA- ESL 

42% 

COMPASS - Math, Reading, Writing and ESL 

33% 

CTEP - Reading and writing 

16% 

MDTP- Math 

36% 

Locally developed multiple choice - Math, Reading, 
Writing and ESL 

14% 

Locally developed performance test - Math, Reading, 
Writing and ESL 

16% 


Educational Background Information 

Percent of respondents using this 

Total 48 of 59 (81%) colleges used educational background 

Length of time out of school 

60% 

Highest level of educational attainment 

64% 

High school GPA 

79% 

General Proficiency in math 

44% 

Grade in last math class completed 

88% 

Highest math course completed 

90% 

Length of time since last math class 

79% 

General Proficiency in reading and writing 

44% 

Grade in last English class completed 

81% 

Highest English course completed 

52% 

Number of years of high school English 

63% 
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Plans Goals, and Experience 

Percent of respondents using this 

Total 44 of 59 (75%) colleges used educational background 

Student's Ed Goals 

80% 

Student's choice of major 

68% 

Number of units student plans to enroll in 

77% 

Highest math course student plans to take 

48% 

Time of day attending 

45% 

Students attitude toward studying 

55% 

Planned hours to study 

61% 


College Education 

Percent of respondents using this 

Total 44 of 59 (75%) colleges used educational background 

College GPA 

50% 

College units completed 

50% 

College degree earned (foreign) 

55% 




Other Characteristics 

Percent of respondents using this 

Total 42 of 59 (71%) colleges used educational background 

Age 

55% 

Veteran Status 

60% 

Importance of college to student 

67% 

Importance of college closest to student 

55% 

Hours employed 

67% 

time spent in extracurricular activities 

36% 

Time devoted to family commitments 

38% 

Perseverance with academic challenge 

48% 

Time spent reading English 

48% 

Ease of reading/writing in English 

45% 
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Appendix B: Complete Table oe Multiple Measures 


Measure 

Pros 

Cons 

Additional notes 

Assessment using a standardized test 



Specific placement tests taken 

Standardized and 

A single high stakes 

Requires cut score validation. 

prior placement in college 

comparable among 

test may not reflect 


courses in English, reading, 

students that have 

all the student's skills 

Must be approved by CCCCO. 

ESL or math e.g., Accuplacer, 
MDTR CASAS. See California 
Community College Chancellor's 
Officer (CCCCO) Approved 
Assessment Instruments (2013).^ 

taken the test. 

and abilities and may 
introduce testing bias. 
Accuracy may vary 
greatly, influenced by 
curriculum and test 
content alignment. 

See CCCCO Approved Assessment 
Instruments (2013). 

California Standards Test (STAR 

Standardized and 

A single high stakes 

New testing focused on the 

test). 

comparable among 

test which may not 

common core will not include 


students that have 

reflect all the student's 

the EAP. 

This includes the EAP; early 
assessment program developed 

taken the test. 

abilities. 

Long Beach City College (LBCC) 

by the California State University 

EAP provides early 


study indicated little correlation 

system for early indicators for 

indicators to allow 


with actual outcomes in LBCC 

high school juniors 

high school students 
time to remediate 
before enrollment in 
college. 


courses. 

Prior educational achievement 




Self-reported high school or 

Provides an aggregate 

May not be reported 

May be affected by the 

college GPA 

measure of student 
achievement and 
motivation. 

accurately. 

recency with which the skills or 
knowledge were acquired and the 
geographic location of the high 
school 

High school (transcript) 

Provides a 

Concerns about high 

In a narrow study at LBCC 


documented 

school GPA inflation 

involving one high school district, 

College transcript 

aggregate measure of 

and significant 

researchers reported a high 


student achievement 

differences among high 

correlation with course success 


and motivation. 

school practices. 

based upon high school course 
grades as the major placement 
measure outcomes. Some LBCC 
faculty question these reported 
results and note that placement 
through high school grades 
raised other significant issues that 
may have negatively impacted 
instruction. 


' http://extranet.cccco.edu/Portals/l/SSSP/Matriculation/ Assessment/ ApprovedAssessmentInstrumentsFall2013.pdf 
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Measure 

Pros 

Cons 

Additional notes 

Highest level of math, English 
or ESL 

Provides discipline 
specific achievement. 

Same as above. 

May be affected by the recency 
and the alignment of the high 
school curriculum with the 
college curriculum. 

Any senior level English and/or 
math course 

English and math 
courses taken in the 
senior year of high 
school, when they 
are not required, 
are evidence of 
motivation. 

This measure does 
not indicate a specific 
course level, but merely 
the completion of any 
a course in math or 
English. 


Advancement Placement (AP) 
course completion or AP scores 
or International Baccalaureate 
(IB) coursework, or test scores 
in HS 

Provides discipline 
specific information 
about motivation and 
achievement. 

Same as above. 

Same as above. 

Demographics and socioeconomic status 

Age 

In some disaggregated 
data, age is correlated 
with success or lack of 
success. 

This data are not 
consistent across ages 
or among specific 
discipline success. 

Students claiming skills upgrade 
as their educational goal have 
very different outcomes by age 
than the general population. 

Units planned 

Evidence suggests that 
full-time status results 
in better success and 
part-time status results 
in poorer success. 

Evidence in this area 
is limited and may not 
take into consideration 
other factors regarding 
full-time vs. part-time 
performance such as 
the economic or family 
conditions that allow 
full-time enrollment. 

Planned units within a specific 
time frame are not always 
represented of the actual units 
completed. 

Hours employed 

Directly relates to 
student effort in 
CCSSE where limited 
work hours positively 
influence success. 

This measure is 
self-reported and 
may therefore raise 
questions of accuracy. 
Work hours and times 
may vary having a 
positive effect during 
one period of time and 
a negative effect at 
another period of time. 

The direct correlations are 
unknown and may involve many 
other factors and variables such 
as the actual hours and days 
worked or the relationship of 
work to the courses and content 
studied. 
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Measure 

Pros 

Cons 

Additional notes 

First Generation status (highest 
level of parental education) 

Provides insight into 
student support needs 
but in no way assumes 
or predicts a student's 
ability. 

Does not directly 
relate to skills and 
knowledge. This 
measure may reflect 
more on institutional 
support and outreach 
than on individual 
success. 

Many data sources indicate 
when student capability is 
evident (even measured through 
diagnostic testing), success is 
more influenced by support than 
student performance. CCCs are 
currently examining more data to 
understand this variable. 

Socioeconomic status (federal 
Pell grant or Board of Governors 
Fee Waiver recipient) 

Provides insight into 
student support needs. 

Does not directly 
relate to skills and 
knowledge. 

Complex variable influenced 
by work hours and family 
responsibilities, but data 
indicates lower income students 
have a more difficult time with 
completion. 

Parenthood or direct family 
responsibilities 

Provides insight into 
student support 
needs and student 
responsibility. 

Must be self-reported 
and may therefore raise 
questions of accuracy; 
potential time 
management issues. 

Direct correlations are unclear. 

Some data in Science, Technology, 
Engineering, and Mathematics 
(STEM) indicates single parents 
have better success rates in career 
technical courses. 

Understanding motivational and maturity factors of students is complex and involves factors that have a potential of 
encouraging bias. The CCRC paper Do High-Stakes Placement Exams Predict College Success? (Scott-Clayton) asserts, 
"demographic variables such as gender, age, race, or ethnicity, which may have predictive value but would be unethical 
to consider in placement decisions." (p. 25) Concerns exist that these socioeconomic factors may result in biased or 
discriminatory placement. On the other hand, these factors may provide useful prognostic data concerning the student 
service needs that then result in greater success. 

Affective measures 

Motivation (self-reported) 

Provides students' 
own assessment of 
motivation level. 

Self-reported 
information is very 
subjective. 

Level of honesty may vary and 
motivation may vary based on 
circumstances; may fluctuate 
during the term. 

Declaration of a major 

Research indicates this 
correlates with success. 

Declaration of a major 
is only helpful if it is a 
committed declaration. 

Accuracy issues arise in the 
selection, recording, and 
changing of a major. 

Other measureable factors that contribute to academic success 
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Measure 

Pros 

Cons 

Additional notes 

Degrees, classifications or 
certifications 

Completion of a series 
of study provides an 
aggregate measure of 
the student's ability to 
complete academic 
goals in the same way 
a HS diploma provides 
information relevant 
to attendance and 
overall life success. 

Not all degrees and 
certificates provide 
the same level of 
information. 

Some military training or 
proprietary schools have a lower 
degree of rigor or focus on skills 
without background. 

Internships 

Internships correlate 
well with success 
where content and 
skills are aligned. 

The breadth and 
depth of the student's 
abilities are difficult to 
truly assess in order to 
place them in a specific 
course or program. 


Computer skills 

Predictor of success. 

This factor is 
commonly referred 
to as the “digital gap" 
and may measure 
socioeconomic status 
rather than student 
abilities. 

Use of this as a multiple measure 
may result in bias, preferentially 
advantaging those with more 
money and access to personal 
computers and technology. 

Program of study - Student 
education plan (SEP) that begins 
in 8th grade or HS and extends 
through college 

Correlated with 
success. 

Depends upon rigor, 
content, and alignment 
of the plan with 
realistic student goals. 

SEPs vary widely with regard to 
quality and investment. 

Employment history 

Direct knowledge and 
skills correlation results 
in great success. 

Lack of alignment 
of employment and 
course expectations 
may not correlate or 
contribute to course 
success. 

May contribute to maturity and 
better success in many situations 
not just course taking. 
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Appendix C 

The following scenarios show very different local methods of establishing placement processes and illustrate 
how discipline experts and counseling faculty and researchers can work with the local senate to establish the 
multiple measures and the process to place students accurately into either prerequisites or target courses. 

Scenario i 

Discipline experts are actively involved in investigating the role placement plays in student success. Faculty 
regularly request that cut scores and student success by course be analyzed or researched. Over the last 
ten years, student success rates have risen significantly in all courses in one particular academic sequence. 
Discipline experts have difficulty understanding the counselors’ role in placement decisions and decide to 
review the multiple measures process. The discipline experts determine that the best way to make the use 
of multiple measures more consistent is to include them as part of the assessment test in the form of five 
addifional mulfiple choice quesfions meant to assess student motivation and prior achievement. This new 
assessment practice means that counselors do not collect additional multiple measures after the assessment 
test is given since the multiple measures are included as part of the assessment for all students. Counselors 
do not use multiple measures to override assessment test decisions and self-reported measures are worth 
very few points in the total assessment score and placement process. 

Scenario 2 

Each department that uses a test to place students into courses is asked to generate what they intend their 
multiple measure process to be. At this point, each department has an entirely separate set of desired measures. 
Next, the senate works with the counselors and the college committee structure to hold a Multiple Measures 
Retreat. At this retreat the relevant Title 5 sections are discussed and the intent of multiple measures is 
presented. At the meeting, the participants determine to create a single survey — called the Student Success 
Inventory — that can be given prior to assessment and that will address as many multiple measure points as 
possible. Each assessment test might only use a few questions from the survey, but collectively the survey is 
both concise and can be used from multiple areas. 

The survey is fully vetted by the local Assessment and Matriculation Committee and is approved by the 
local academic senate. The role of counseling is the subject of much discussion, but ultimately the college 
leaves the decision to each individual department. Some, such as mathematics, are comfortable with giving 
counselors considerable latitude to use their professional judgment, but other departments, such as Learning 
Assistance, desire a specific points-based model fhaf leaves little authority to the individual evaluator. 

The scenarios above demonstrate that all colleges have different processes for establishing local processes. 
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Appendix D 

Questions to ask to begin your placement process using multiple measures. 

What is your college’s placement process? 

Does your college have an assessment and matriculation committee? 

Is your college’s placement process based on data? 

Who decides the placement? 

Who communicates the assessment procedures and the way placement decisions are made? 

How do students find out which multiple measures are being used to assess their level of knowledge 
and skill and how those multiple measures will be analyzed? 

Is your college’s process consistent? 

Is it used on aU students? 

Do your college’s multiple measures have predictive validity? 

Has the college determined the placement decisions results are successful regardless of who makes 
them? 

Who was involved in deciding which multiple measures are used on your campus? 

Does your college have a local selection of validated measures policy and data? 

Does the college use a periodic review of multiple measures assessment policies? 

Are discipline experts and counseling faculty given information on why certain multiple measures 
have been selected for use at the colleges? 

Are discipline experts and counseling faculty given information on the role that multiple measures 
can play in accurate placement? 

Does your college use interviews? If so, has there been a review of the placement decisions and the 
success values to determine whether or not the interviews improve the process? 

Is your college communicating with the local high schools? 

Have you talked to other local colleges in your area to determine what placement processes they 
are using? 
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